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ABSTRACT 

World Wide Web-based component for a required freshman seminar at the 
Pennsylvania State University College of Agricultural Sciences. Students 
(n=170) were given a pre-test to assess their access to, knowledge of, and 
proficiency with computers. The pre-test was designed to address four broad 
categories: computer use, ability, and perceptions; Internet perceptions and 
use; communication preferences; and demographic information. This formative 
evaluation assessed student needs and helped the project team to continue to 
develop course content for the semester. A summative evaluation was given at 
the end of the semester to ascertain students' perceptions of web-based 
assignments and needed changes for future courses. In addition, one faculty 
member from each section participated in a phone survey, answering questions 
about their experiences with the course. It was concluded that using a 
computer-based asynchronous teaching model is quite different from the more 
traditional model and requires special considerations; practitioners should 
incorporate formative and summative evaluations to enhance learner 
satisfaction, to ensure goal attainment, and to demonstrate accountability. 
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Distance learning methodologies support a wide variety of academic programs for 
residential and off-campus students. These delivery technologies, including audio and video 
teleconferencing, computer conferencing, and web-based instruction are changing the way 
students interact with subject matter and faculty. In addition to enhancing traditional 
learning practices, distance learning technologies affect how students engage the global 
community through on-line resources. Internet site design and user compatibility provide a 
starting point to integrate computers into instruction. 

Web-based instruction, for example, is growing faster that any other instructional 
technology (Crossman, 1997). With a computer connection, students and faculty use the web 
to exchange information and access resources from around the world. The popularity of 
web-based instruction is attributed to its convenience and flexibility of access (Daugherty 
and Funke, 1998). 

The innovative nature of distance education methodologies demands close examination 
regarding the issues and practices relevant to educational quality and integrity. Reeves and 
Reeves (1997) concluded that there are many issues relevant to the web that have to be fully 
investigated for their pedagogical soundness. Web resources are viewed as a means by 
which to keep courses current, however, accuracy and timeliness plague sites that are not 
regularly updated. 

While some faculty embrace the challenge to incorporate new technologies into the learning 
environment, others are overwhelmed by them (Collis, 1993 ). Dillon and Walsh (1992) 
found that faculty involved in distance education acquire more positive attitudes as their 
experience with distance education increase. Herther (1997) suggests that the quality of 
learning through distance education be evaluated before web-based instruction is subsumed 
and adopted into university practices. 

Evaluation is an integral part of course delivery and development. Cost-benefit, learner 
satisfaction, goal attainment, and accountability require faculty to gather and submit 
feedback on the effectiveness of course process and content. Evaluation studies provide 
timely feedback and constructive criticism to the developers and designers using 
information technology while the curriculum is still evolving (Collis, 1993). Positive 
evaluations encourage administrative support of policies, practices, and infrastructure 
relevant to distance education. Furthermore, insights gained broaden faculty understanding 
of the commitments necessary to develop quality programs that enhance the traditional 
learning environment. 
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Collis (1993) suggests that distance education projects are marginally evaluated. 
Furthermore, when they are evaluated, the evaluations focus on either client satisfaction or 
factors correlated with learner persistence or attrition. This paper focuses on participants' 
feedback about the content and relevance of web-based instruction. Students and faculty 
were queried about product and process. The importance of faculty and learner feedback in 
furthering the distance education mission is stressed. Formative evaluations were used to 
provide information about improving the course. Summative evaluation were carried out to 
make judgments about the basic worth of incorporating web-based technology into the 
freshmen seminar. 



Integrating Distance Education Technologies 

Beginning fall 1999, the Pennsylvania State University will require all entering students to 
take a freshmen seminar. The purpose of the seminars is to support the transition of 
students to the University environment. Penn State's College of Agricultural Sciences offers 
Be a Master Student! (AG 150) as a two-credit course to entering freshmen. The course focuses 
on 1) facilitating the student's transition to the university community, and 2) increasing each 
student's understanding of the issues and opportunities in the agricultural sciences. 

To address an identified need, a web-based component was added to the AG 150 curriculum 
in fall 1997. The intent was to use communications technology to enhance the agricultural 
sciences component of the curriculum, making the course available to Penn State students 
and non-students throughout the Commonwealth. The goal was 1) to develop web-based 
resources that would facilitate the exploration of current issues in the agricultural sciences 
and their associated resources at Penn State, and 2) to develop a series of activities or lesson 
plans for students and faculty teaching AG 150 to integrate into the curriculum. 

Before this project, computer-based instruction was not available to faculty teaching AG 150. 
The website supplements the classroom experience by providing a solid core of common 
resources across sections and campuses as well as the opportunity for students and 
instructors to interact electronically outside of class. In addition, faculty could draw course 
assignments from a plethora of web-based resources. 

Project Development 

Technology-based instruction requires much planning and collaboration. Integration of 
technology resources for AG 150 was a complex task. It required the synergy and patience of 
faculty, support staff, and students. The website developed to supplement instruction in the 
College's freshmen seminar was not created by faculty currently teaching the course. Two 
faculty members from the College of Agricultural Sciences team taught each of the nine 
sections. Faculty, often independent and accustomed to teaching autonomously, were 
required to work in teams to collaboratively develop course materials. Thus, it was 
imperative that tools and other opportunities be developed to demonstrate to the teaching 
faculty how the site could support their instructional objectives. 

Web-based instruction is a relatively new methodology in higher education, and many 
issues still need to be addressed. Lack of faculty incentives, limited access to technology, and 
insignificant support can hinder successful delivery (Bowen and Thomson, 1994). Lessons 
learned through formative and summative evaluations of the instructional technology 
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portion of the AG 150 seminar provide a road map for faculty pursuing web-based 
instruction. 

Because the University has made the freshmen seminar a requirement, AG 150 has become a 
model for other colleges to follow. This prototype includes among other things — World 
Wide Web-based instruction with virtual tours, career path designs, faculty interviews, and 
links to many relevant resources on the Internet. The site http:/ / www.cas.psu.edu/ docs/ 
CASOVER/AG150.index.htm is relevant to anyone interested in agricultural issues. 

Evaluation Process 

To test the goals for the project and to learn even more about this new learning approach, it 
was agreed from the outset to collect as much data as reasonable. The population for the 
study was all of the students in AG 150 Be a Master Student! during fall semester, 1997. 
Because the number of students in AG 150 was small (N = 170), a census was used. Students 
were given a pre-test at the beginning of the semester to assess their access to, knowledge of, 
and proficiency with computers. The pre-test instrument used was designed to address four 
broad categories: 1) computer use, ability, and perceptions; 2) Internet perceptions and use; 
3) communication preferences; and 4) demographic information. The formative evaluation 
assessed student needs and helped the project team continue to develop course content for 
the semester. 

During the semester, faculty used the web-site to supplement in-class lectures and provide 
resources for assignments. Three of the nine sections specifically used the AG 150 web site. 
One instructor maximized the AG 150 site by using it in another course. Instructors 
indicated that helping students determine how to evaluate the credibility of web-based 
resources now needed to be incorporated into their instruction. 

A summative evaluation was given at the end of the semester to ascertain students' 
perceptions of web-based assignments and needed changes for future courses. Students 
were queried on using the Web for AG 150 and other courses. Among the 170 students 
registered, 142 usable, completed post-test questionnaires were collected, an 84% response 
rate. One of the faculty members from each section participated in a phone survey, 
answering questions about their experiences with the course. 

Findings 

The formative evaluation queried students about course content, expectations, and use of 
the World Wide Web. The survey findings indicated that generally, freshmen access to and 
knowledge of computers is increasingly. Of the 142 students who were surveyed during fall 
1996, 57% owned computers. Among those who owned their computers, 46% (n = 66) 
indicated that their computers were connected to the Internet. Among the two-thirds (n = 
108) who owned their computers in fall 1997, 72% indicated that their computers were 
connected to the Internet. Also fall 1997, three-quarters of the students (74.6%) responded 
that they were already using computers at least once a day. Almost half of the students 
(45%) stated that they used the Web for class assignments. Forty-four percent indicated that 
a course that required a web-based supplement appealed to them. The students also 
indicated interest in learning about job opportunities, internships, and University resources. 
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In response to student surveys, faculty integrated additional content on career opportunities 
into the course before the semester's end. 

The summative evaluation addressed learner satisfaction with the technology. When asked 
what they liked most about using the Web, two-thirds (n = 93) of the students expressed that 
they liked its convenience and wealth of information. Typical reasons students gave for 
using the Web: 

♦♦♦ "It's easy to get a lot of information without leaving home. " 

❖ "It's a hands-on approach to learning." 

♦♦♦ "I think it is important to be able to use the Web, because it is such a prevalent means 
of communication." 

In addition, students expressed a concern regarding the reliability of web-based information. 
When prompted about the accuracy of web-based information, 57% of the 142 students 
indicated that they considered the source to determine if information is credible. 

This freshmen seminar is one of many projects to integrate distance education technologies 
into University curricula. In every section, students were expected to submit at least some, if 
not all, assignments via e-mail. The results indicated that students who do not own personal 
computers must be considered when designing technology-based instruction. Instructors 
consistently commented on the increasing computer literacy among students during the past 
two years. Specifically noted was that students with the fewest computer skills were non- 
University Park (main) campus based. Faculty involved in recruitment for the College in 
addition to the freshmen seminar recognized the Web's potential as a recruitment tool 
among secondary students as well as among those in higher education who are outside the 
agricultural sciences. For the site to be used in this way, faculty noted that placement on the 
College's homepage becomes crucial. 



Conclusion 

The project revealed that using a computer-based, asynchronous teaching model is quite 
different from the more traditional model and requires special considerations. Although 
entering freshmen are expected to be increasingly computer literate, students enter the 
university with varying levels of expertise. To optimize the students' educational 
experiences, faculty will need to be aware of the competencies students bring in order to 
maximize their learning opportunities. 

Furthermore, it is important to note that access to technology is not a significant incentive for 
faculty to embrace new teaching methodologies. Relevance to subject matter, timeliness of 
information, and facilitation of instructional objectives are required for successful integration 
of web-based resources. 

While many students enter the university with Internet experience, some faculty are still 
learning how to incorporate instructional technologies into learning opportunities. Faculty 
need administrative support and opportunities to develop effective technology-based 
courses. Faculty must openly communicate with students and each other throughout the 
learning experience to develop curricula that effectively use distance education technologies. 
Web-based information is easily accessible and convenient, however, its reliability and 
accuracy can be questionable. Faculty that incorporate web-based instruction into the 
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curriculum should develop ways for students to test the accuracy of information found on 
the web. 

The evaluation process is key in the communications process. Identifying relevant issues and 
problems during the course provides opportunities to develop solutions for quality 
educational programs. Student feedback during the learning process helps to shape the 
course and improve the learning experience. Practitioners that seek to integrate distance 
education methodologies into existing curriculum should incorporate formative and 
summative evaluations to enhance learner satisfaction, to ensure goal attainment, and to 
demonstrate accountability. 
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